Unified dual for bi-class SVM approaches

SVM theory was originally developed on the basis of a separable binary classification problem, and other approaches have been later introduced. In this paper is demonstrated that all these approaches admit the same dual problem formulation. Let Z = ((x1, y1), . . . , (xn, yn)) = (z1, . . . , zn) ∈ (X × Y)n be a training set, where X is the input space and Y = {θ1, θ2} = {−1, +1} the output space. Let φ : X → F be a feature mapping and x def = φ(x) ∈ F be the representation of x ∈ X . A linear classifier, fw(x) = 〈x,w〉− b is sought in the space F , with fw : X → R, and outputs are obtained as hw(x) = sign(fw(x)). Let Z1 and Z2 be the patterns belonging to the classes labelled as {+1,−1}, respectively, and ni = #Zi, then it is defined, β = minzi∈Z1 〈xi,w〉, α = maxzj∈Z2 〈xj,w〉 so that when classes are linearly separable in the feature space, then α ≤ β. Hence, classifier w with the largest geometrical margin on a given training sample Z is wSV M def = arg max w∈F ;α,β∈IR β − α ‖w‖ Several approaches can be derived to translate this problem in an optimization problem (see Table 1). Main contribution of this work is that all the existing QP problem formulations are the same when dualized, as established by the following theorem. Theorem 1 Dual expression of the optimization problems (1), (2), (3) and (4) can be formulated as: min u∈IRn1 ,v∈IRn2 1 2 ∥∥∥∥∥ n1 ∑